Neural Networks for Pattern Classification and Universal Approximation

LIAO, YI. Neural Networks for Pattern Classification and Universal Approximation (Under the direction of Dr. Shu-Cherng Fang and Dr. Henry L. W. Nuttle). This dissertation studies neural networks for pattern classification and universal approximation. The objective is to develop a new neural network model for pattern classification, and relax the conditions for Radial-Basis Function networks to be universal approximators. First, the problem of pattern classification is introduced, which is followed by a brief introduction of three popular nonlinear classification techniques, that is, Multi-Layer Perceptrons (MLP), Radial Basis Function (RBF) networks, and Support Vector Machines (SVM). Then, based on the basic concepts of MLP, RBF and SVM, a new neural network model with bounded weights is proposed, and some experimental results are reported. Later, the problem of universal approximation by neural networks is introduced, and the researches on ridge activation functions and radial-basis activation functions are reviewed. Then, the relaxed conditions for RBF networks to be universal approximators are presented. We show that RBF networks can uniformly approximate any continuous function on a compact set provided that the radial basis activation function is continuous almost everywhere, locally essentially bounded, and not a polynomial. Some experimental results are reported to illustrate our findings. The dissertation ends with the conclusion and future research. Neural Networks for Pattern Classification and Universal Approximation